The reliability of individual software modules is an ever increasing factor in determining overall system reliability as the size, scope, complexity and degree of interaction among modules of software systems have increased. Furthermore, given business dependence on software reliability, estimation of software performance has become critical to the successful development and deployment of large-scale software systems. Accordingly, software engineers have been striving to measure software quality. However, this effort to measure software quality has often times been frustrated by the subjective nature of the activity. In most instances, what is measured is not quality per se, but some indirect measurement that measures some manifestation of quality. However, software engineers often times do not know the precise relationship between the measurable characteristics of the software and the quality of the software. Various methods have been devised by those practicing in the art to quantify software quality.
As an example, the U.S. Air Force Systems Command has developed a number of quality indicators that are based on measurable design characteristics of a computer program. One such indicator is called the design structure quality index (DSQI). The DSQI is calculated from the following seven measured characteristics of a software system:
S.sub.1 =the total number of modules defined in the software architecture. PA1 S.sub.2 =the number of modules whose correct function depends on the source of data input or that produces data to be used elsewhere. PA1 S.sub.3 =the number of modules whose correct function depends on prior processing, PA1 S.sub.4 =the number of database items (includes data objects and all attributes that define objects) PA1 S.sub.5 =the total number of unique database items PA1 S.sub.6 =the number of database segments (different records or individual objects) PA1 S.sub.7 =the number of modules with a single entry and exit PA1 D.sub.1 =if the architectural design was developed using a distinct method then D.sub.1 =1 otherwise D.sub.1 =0 PA1 D.sub.2 =is a measure of module independence and equal S.sub.2 /S.sub.1 PA1 D.sub.3 =is a measure of modules not dependent of prior processing and equals 1-(S.sub.3 /S.sub.1) PA1 D.sub.4 =is a measure of database size and equal 1-(S.sub.5 /S.sub.4) PA1 D.sub.5 =is a measure of database compartmentalization and equals 1-(S.sub.6 /S.sub.4) PA1 D.sub.6 =is a measure of module entrance and exit characteristics and equals 1-(S.sub.7 /S.sub.1). PA1 1 Collect and categorize information about software defects. PA1 2 Attempt to trace each defect to its underlying cause. PA1 3 Under the principal that eighty percent of the defects can be traced to 20 percent of the causes, isolate the 20 percent vital few causes. PA1 4 Once the vital few causes have been identified, move to correct the problems that have caused the defects.
Once the above values are measured, the following intermediate values can be computed:
Using these intermediate values, the DSQI can be computed using the following equation: EQU DSQI=.SIGMA.W.sub.i D.sub.i
where i=1 to 6, w.sub.i is the relative weighting of the importance of each of the intermediate values, and .SIGMA.W.sub.i =1. Weighting is often determined by a software engineer and is usually based on the professional judgement of the engineer. Therefore, there is a need in the art to objectively determine the value (weighting) of metrics such as they relate to software quality.
The value to the software engineer in computing the DSQI is to compare past designs to a design currently under development. If the DSQI is significantly lower than the average for past designs it is usually taken as an indication that further design work and review is indicated. However, it has yet to be objectively determined if the DSQI is an effective predictor of software quality. In addition, it doesn't take into account changes in the development process that may affect software quality. Therefore, there is also a need for objectively determining the value of such metrics as they vary between different software development environments over time.
The software engineering community has developed other metrics or indices for measuring software in an attempt to quantify and predict the quality of a new software product. IEEE Standard 982.1-1988 suggests a software maturity index that provides an indication of the stability of a software product based on changes that occur for each release of the product. Another measurement is a complexity measure proposed by Mr. Thomas McCabe (McCabe, T. J., and Butler, C. W., "Design Complexity Measurement and Testing," CACM, Vol. 32, no. 12, December 1989, pp 1415-1425). McCabe defines a software complexity measure that is based on the cyclomatic complexity of a program graph for a module. Experimental studies indicate a distinct relationship between McCabe's metric and the number of errors existing in source code as well as the time required to find and correct such errors. However, McCabe's metric does not objectively quantify the predicted performance of a software module.
More recently, statistical quality assurance methods have been incorporated into the software engineering process. The traditional prior art approach to statistical quality assurance generally is as follows:
The key to conducting a statistical analysis is tracing, or correlating, a measurable statistic to a software characteristic. As an example, software engineers and analysis may examine errors in design logic or incomplete or erroneous specifications as they relate to serious defects. However, the relationship is usually determined in an empirical study using a limited number of factors, which when determining a statistic then remains static. What is lacking in the prior art is a dynamic methodology and tool that can examine all measurable characteristics of a specific software system and its development environment and identify those characteristics that are high risk indicators of quality problems.